Vehicle backup safety mapping

ABSTRACT

Method and apparatus are disclosed for vehicle backup safety mapping. An example vehicle includes a display, a rear-view camera, and a processor. The processor generates a three-dimensional model of space behind the vehicle based on images from the rear-view camera The processor also generates an overlay based on the three-dimensional model. The overlay includes representation of objects not in the field of view of the rear-view camera. Additionally, the processor displays, on the display, the images from the rear-view camera and the overlay.

TECHNICAL FIELD

The present disclosure generally relates to vehicle rear-view camerasand, more specifically, vehicle backup safety mapping.

BACKGROUND

When backing up and using a rear backup camera, a driver can becomedisoriented to the position of the car relative to other static objects(e.g. parked vehicles, support columns, barriers, etc.), especially intight parking spots requiring repeat turns and reversing. The rear viewcamera has a limited field of view (FOV) that is limited by the physicalposition of the vehicle. As a result, other object, such as other parkedvehicle, may not always be in view. For example, the fish eye lens willhave a large FOV, typically 90 to 170 degrees, to present the drive withvision of vehicles to the edges of the car. However, even with a wideFOV, the position and geometry of the vehicle and position of the camerawill create natural blind spots primarily located at the rear cornerbumpers of the vehicle.

SUMMARY

The appended claims define this application. The present disclosuresummarizes aspects of the embodiments and should not be used to limitthe claims. Other implementations are contemplated in accordance withthe techniques described herein, as will be apparent to one havingordinary skill in the art upon examination of the following drawings anddetailed description, and these implementations are intended to bewithin the scope of this application.

Example embodiments are disclosed for vehicle backup safety mapping. Anexample vehicle includes a display, a rear-view camera, and a processor.The processor generates a three-dimensional model of space behind thevehicle based on images from the rear-view camera. The processor alsogenerates an overlay based on the three-dimensional model. The overlayincludes representation of objects not in the field of view of therear-view camera. Additionally, the processor displays, on the display,the images from the rear-view camera and the overlay.

An example method to assist a reverse moving vehicle includes generatinga three-dimensional model of space behind and/or to the sides of thevehicle based on images from a rear-view camera and generating anoverlay based on the three-dimensional model. The overlay includes across-section of the model representing edges of surfaces within themodel. The method also includes displaying, on a center console display,the images from the rear-view camera and the overlay. Additionally, themethod includes providing an alert via an infotainment system when (i)one of the surfaces in the model is identified as belonging to a hazardor (ii) when the vehicle comes within a threshold distance to one of thesurfaces according to the model.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, reference may be made toembodiments shown in the following drawings. The components in thedrawings are not necessarily to scale and related elements may beomitted, or in some instances proportions may have been exaggerated, soas to emphasize and clearly illustrate the novel features describedherein. In addition, system components can be variously arranged, asknown in the art. Further, in the drawings, like reference numeralsdesignate corresponding parts throughout the several views.

FIG. 1 illustrates a vehicle operating in accordance with the teachingsof the disclosure.

FIGS. 2 and 3 illustrate displays on an infotainment head unit of thevehicle of FIG. 1.

FIG. 4 is a block diagram of electronic components of the vehicle ofFIG. 1.

FIGS. 5 and 6 are flowcharts of methods to map obstacles behind thevehicle of FIG. 1 while the vehicle is backing up, which may beimplemented by the electronic components of FIG. 4.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

While the invention may be embodied in various forms, there are shown inthe drawings, and will hereinafter be described, some exemplary andnon-limiting embodiments, with the understanding that the presentdisclosure is to be considered an exemplification of the invention andis not intended to limit the invention to the specific embodimentsillustrated.

As described below, a vehicle backup safety mapping system uses inputfrom (a) a rear-view camera (b) other sensors (e.g., ultrasonic sensors,RADAR, LiDAR, etc.) and/or cameras (e.g., side view mirror cameras,etc.) and (c) information (e.g., from electronic control units (ECUs))regarding wheel orientation and rotation to generate a representation ofthe three-dimensional space behind the vehicle. The motion of thevehicle is tracks through the three-dimensional space to provide to adriver the representation of the space behind the vehicle that is notwithin the current view of the rear-view camera. When the gear shifteris positioned into reverse, the system analyzes images captured by therear-view camera and data captured by the other sensors to determine thethree dimensional structure of the surroundings of the vehicle withrespect to the vehicle. As the vehicle moves in reverse, the systemcontinues to update that three dimensional structure and tracks thevehicle's movement through the space. A representation of the threedimensional space is overlaid onto the image captured by the rear-viewcamera displayed on a display of the infotainment head unit. The overlayincludes representations in the three dimensional space that are notcurrently in the view of the rear-view camera. These representations mayinclude an overhead, isometric, or another perspective of the threedimensional space. In some examples, the representations includecoloring to signify different properties of certain portions of thethree dimensional space. Additionally, the system provides proximityalerts to the driver when the vehicle approaches obstacle(s) that arenot in the current view of the rear-view camera or other sensors viatracking said objects in space with movement of the vehicle.

FIG. 1 illustrates a vehicle 100 operating in accordance with theteachings of the disclosure. The vehicle 100 may be a standard gasolinepowered vehicle, a hybrid vehicle, an electric vehicle, a fuel cellvehicle, and/or any other mobility implement type of vehicle. Thevehicle 100 includes parts related to mobility, such as a powertrainwith an engine, a transmission, a suspension, a driveshaft, and/orwheels, etc. The vehicle 100 may be non-autonomous or semi-autonomous(e.g., some routine motive functions controlled by the vehicle 100). Inthe illustrated example the vehicle 100 includes sensors 102 a-102 e,cameras 104 a and 104 b, an infotainment head unit 106, and an on-boardcomputing platform 108.

Sensors may be arranged in and around the vehicle 100 in any suitablefashion. The sensors may be mounted to measure properties around theexterior of the vehicle 100. Additionally, some sensors may be mountedinside the cabin of the vehicle 100 or in the body of the vehicle 100(such as, the engine compartment, the wheel wells, etc.) to measureproperties in the interior of the vehicle 100. For example, such sensorsmay include accelerometers, odometers, tachometers, pitch and yawsensors, microphones, tire pressure sensors, and biometric sensors, etc.In the illustrated example, the sensors 102 a-102 e include ultrasonicsensors 102 a, RADAR 102 b, LiDAR 102 c, wheel speed sensors 102 d,and/or wheel angle sensors 102 e.

The ultrasonic sensors 102 a, RADAR 102 b, and/or LiDAR 102 c are usedto detect objects (e.g., other vehicles 110, pillars, barriers,pedestrians 112, etc.) in the vicinity of the vehicle 100. The datacaptured by the ultrasonic sensors 102 a, RADAR 102 b, and/or LiDAR 102c is used to generate a three dimensional map of the area around thevehicle 100. The ultrasonic sensors 102 a, RADAR 102 b, and/or LiDAR 102c are positioned on the vehicle 100 to detect the objects in the spacearound the vehicle 100 not captured by the rear-view camera 104 a. Thewheel speed sensors 102 d and/or wheel angle sensors 102 e are used todetermine the location of the vehicle 100 within the three dimensionalmap and determine the corresponding portion of the three dimensional mapthat is viewable from the rear-view camera 104 a.

The cameras 104 a and 104 b capture images of the area around thevehicle 100. The images are used to detect and/or identify objectsaround the vehicle 100. The rear-view camera 104 a is mounted on therear of the vehicle 100. The area behind the vehicle 100 that thecameras 104 a can capture is defined by a field of vision (defined bythe viewable angle θ), distortion coefficients, focal lengths, andoptical centers. While the transmission of the vehicle 100 is inreverse, the images captured by the rear-view camera 104 a are displayedon the infotainment head unit 106. The side-mirror cameras 104 b arepositioned to capture images to detect objects that are not captured bythe rear-view camera 104 a.

The infotainment head unit 106 an interface between the vehicle 100 anda user. The infotainment head unit 106 includes digital and/or analoginterfaces (e.g., input devices and output devices) to receive inputfrom the user(s) and display information. The input devices may include,for example, a control knob, an instrument panel, a digital camera forimage capture and/or visual command recognition, a touch screen, anaudio input device (e.g., cabin microphone), buttons, or a touchpad. Theoutput devices may include instrument cluster outputs (e.g., dials,lighting devices), actuators, a heads-up display, and/or speakers. Theinfotainment head unit 106 includes hardware (e.g., a processor orcontroller, memory, storage, etc.) and software (e.g., an operatingsystem, etc.) for an infotainment system (such as SYNC® and MyFordTouch® by Ford®, Entune® by Toyota®, IntelliLink® by GMC®, etc.). Theinfotainment head unit displays the infotainment system ZZZ on, forexample, a center console display 114. The center console display is adisplay (e.g., a liquid crystal display (“LCD”), an organic lightemitting diode (“OLED”) display, a flat panel display, a solid statedisplay, etc.) that, in some examples, includes a touchscreen. When thetransmission of the vehicle 100 is in reverse, the center consoledisplay 114 displays an interface (e.g., the interface 200 of FIG. 2 orthe interface 300 of FIG. 3 below) that provides the view of therear-view camera 104 a and an overlay of a representation of the threedimensional space behind the vehicle 100.

The on-board computing platform 108 includes processor(s) (e.g., theprocessor 406 of FIG. 4 below), memory (e.g., the memory 408 of FIG. 4below) and/or circuitry to communicate with the sensors 102 a-102 d andthe infotainment head unit 106. In the illustrated example, the on-boardcomputing platform 108 includes a data fuser 116. The data fuser 116receives the data from the sensors 102 a-102 d and the cameras 104 a and104 b to (a) generate a three dimensional representation of the spacebehind the vehicle 100, (b) track the location of the vehicle 100 withinthe three dimensional representation, and (c) generate an overlay toimpose on the image from the rear-view camera 104 a which providesinformation about obstacles that are not visible in the images capturedby the rear-view camera 104 a.

The data fuser 116 monitors for when the transmission of the vehicle 100is placed into reverses to start gathering information from the sensors102 a-102 d and/or the cameras 104 a and 104 b. With the cameras 104 aand 104 b, the data fuser 116 performs image classification orclassification via the 3D information to recognize and/or categorizedobjects (e.g., such as the other vehicles 110, pillars, barriers, thepedestrians 112, etc.). In some examples, the data fuser 116 also usesthe images from the cameras 104 a and 104 b to determine distancesbetween the objects and the vehicle 100. The data fuser 116 creates thethree dimensional map of the area behind the vehicle 100 based onmeasurements from the sensor 102 a-102 c. The data fuser uses themeasurements from the wheel speed sensors 102 d and the wheel anglesensors 102 e to (a) track the location of the vehicle 100 in the area,(b) track the location of the FOV of the rear-view camera 104 a, and (c)determine the physical position and angle of the rear-view camera 104 arelative to the physical geometry of the vehicle 100.

The results of the image classification and the three dimensional mapare combined to generate a point cloud of the area that represents thesurfaces detected by the sensors 102 a-102 d and the cameras 104 a and104 b. The point cloud is a representation of the edges and surfacesdetected in the three-dimensional space behind the vehicle 100. In someexamples, the point cloud also includes the three-dimensional space tothe sides of the vehicle 100 (e.g., as observed by the side-mirrorcameras 104 b and/or sensors 102 a-102 c, etc.). The images captured bythe rear-view camera 104 a are displayed on the center console display114. As shown in FIG. 2 below, a two dimensional cross-section of thepoint cloud (e.g., the representation 212) is overlaid onto the imagedisplayed on the center console display 114 corresponding to (i) the FOVof the rear-view camera 104 a and (ii) a portion of the cross-section ofthe point cloud that is not within, but contiguous with, the FOV of therear-view camera 104 a (e.g., as discussed in connection with FIG. 2below). The data fuser 116 periodically (e.g., every 1 second, every 5seconds, every 10 seconds, every 15 seconds, etc.) repeats this processto update the point cloud and the display based on movement of thevehicle 100 and movement of detected objects. In some example, thefrequency at which the data fuser 116 updates the point cloud and thedisplay is based on the speed of the vehicle 100 and, whether apedestrian 112 is detected, and/or whether two consecutive scans detectsthat an object is moving. For example, if a pedestrian 112 is detected,the data fuser 116 may increase the frequency at which it updates thepoint cloud.

The data fuser 116 provides an alert (e.g., an audiovisual alert, etc.),via the infotainment head unit 106, when the feature detectionidentifies the pedestrian 112 or other hazard (e.g., an object that islow to the ground, such as a bicycle, an animal, etc.). The data fuser116 provides an alert, via the via the infotainment head unit 106, whena distance between the vehicle 100 and the cross-section of the pointcloud is less that a threshold distance (e.g., six inches, one foot,etc.). In some examples, the data fuser 116 provides an alert, via thevia the infotainment head unit 106, when a surface defined by the pointcloud is detected to be in motion. For example, the data fuser 116 maydesignate one color for a portion of the cross-section of the pointcloud that is stationary and another color for the portion of thecross-section of the point cloud that is detected to be in motion.Additionally, in some examples, the data fuser 116 provides an alert,via the infotainment head unit 106, when a portion of the area in behindthat vehicle 100 cannot be observed by the cameras 104 a and 104 b andthe sensors 102 a-102 c. For example, the vehicle 100 may be parked nextto a pillar that obscures behind the vehicle 100 from the sensors 102a-102 c and the cameras 104 a and 104 b. In some examples, the datafuser 116 continues to monitor the area behind the vehicle 100 andprovide alerts until the vehicle 100 is moving forward at a speedgreater than a forward speed threshold (e.g., 5 miles per hour, 10 milesper hour, etc.)

In some examples, the data fuser 116 continually classifies objectsbased on the cameras 104 a and 104 b while the vehicle 100 is movingforward. In some such examples, the data fuser 116 stores the locationsand classifications of the objects classifies as stationary for athreshold distance (e.g., 10 feet, 20 feet, etc.). In such a manner,when the vehicle 100 parks, the stationary objects will already beidentified and classified for when the vehicle 100 is next restarted.

FIG. 2 illustrates an interface 200 displayed on the infotainment headunit 106 of the vehicle 100 of FIG. 1. In the illustrated example, theinterface 200 is dived into a camera view 202 and a projected overheadview 204. The camera view 202 displays the view of the rear-view camera104 a. The projected overhead view 204 includes visible area 206 andnon-visible areas 208. The projected overhead view 204 also includes arepresentation 210 of the vehicle 100 and a representation 212 of thecross-section of the point cloud. The representation 212 of thecross-section of the point cloud is indicative of where the sensors 102a-102 c and/or the cameras 104 a and 104 b have determined boundaries ofoccupied areas behind the vehicle 100. In the illustrated example, aportion 214 of the representation 212 of the cross-section of the pointcloud is displayed in the visible area 206 that corresponds to the imagedisplayed in the camera view 202. Other portions 216 of therepresentation 212 of the cross-section of the point cloud are displayedin the non-visible areas 208 corresponding to boundaries of occupiedareas behind the vehicle 100 of which are not visible because of the FOVof the rear-view camera 104 a. As the vehicle 100 moves and the FOV ofthe rear-view camera 104 a changes, the displayed representation 212 ofthe cross-section of the point cloud changes based on the point cloudand the location of the vehicle 100 tracked with reference to the pointcloud.

FIG. 3 illustrates an interface 300 displayed on the infotainment headunit 106 of the vehicle 100 of FIG. 1. In the illustrated example, theinterface 300 is divided into a visible area 302 and non-visible areas304. Images captured by rear-view camera 104 a are displayed in thevisible area 302. The non-visible areas 304 are representative of areasthat are contiguous with the visible area 302, but are not visiblebecause of the FOV of the rear-view camera 104 a. In the illustratedexample, the objects identified and/or classified based on the imagesfrom the camera 104 a and 104 b and/or the sensors 102 a-102 c areencircled by outlines 306 that represent the boundaries of the objects.When the object extends into the non-visible area 304, the correspondingoutline also extends into the non-visible area 304. In such a manner,the interface 300 includes information about the location of objectsthat cannot be fully seen in the images captured by the rear-view camera104 a. In some examples, when the objects are identified, the interface300 includes labels 308 that provide identifying information of theobject to the occupant(s) of the vehicle 100.

FIG. 4 is a block diagram of electronic components 400 of the vehicle100 of FIG. 1. In the illustrated example, the electronic components 400include the sensors 102 a-102 e, the cameras 104 a and 104 b, theinfotainment head unit 106, the on-board computing platform 108, a powertrain control unit 402, and a vehicle data bus 404.

The on-board computing platform 108 includes a processor or controller406 and memory 408. In the illustrated example, the on-board computingplatform 108 is structured to include data fuser 116. The processor orcontroller 406 may be any suitable processing device or set ofprocessing devices such as, but not limited to: a microprocessor, amicrocontroller-based platform, a suitable integrated circuit, one ormore field programmable gate arrays (FPGAs), and/or one or moreapplication-specific integrated circuits (ASICs). The memory 408 may bevolatile memory (e.g., RAM, which can include non-volatile RAM, magneticRAM, ferroelectric RAM, and any other suitable forms); non-volatilememory (e.g., disk memory, FLASH memory, EPROMs, EEPROMs,memristor-based non-volatile solid-state memory, etc.), unalterablememory (e.g., EPROMs), read-only memory, and/or high-capacity storagedevices (e.g., hard drives, solid state drives, etc). In some examples,the memory 408 includes multiple kinds of memory, particularly volatilememory and non-volatile memory.

The memory 408 is computer readable media on which one or more sets ofinstructions, such as the software for operating the methods of thepresent disclosure can be embedded. The instructions may embody one ormore of the methods or logic as described herein. In a particularembodiment, the instructions may reside completely, or at leastpartially, within any one or more of the memory 408, the computerreadable medium, and/or within the processor 406 during execution of theinstructions.

The terms “non-transitory computer-readable medium” and “tangiblecomputer-readable medium” should be understood to include a singlemedium or multiple media, such as a centralized or distributed database,and/or associated caches and servers that store one or more sets ofinstructions. The terms “non-transitory computer-readable medium” and“tangible computer-readable medium” also include any tangible mediumthat is capable of storing, encoding or carrying a set of instructionsfor execution by a processor or that cause a system to perform any oneor more of the methods or operations disclosed herein. As used herein,the term “tangible computer readable medium” is expressly defined toinclude any type of computer readable storage device and/or storage diskand to exclude propagating signals.

The power train control unit 402 includes hardware and firmware tocontrol the ignition, fuel injection, emission systems, transmissionand/or the brake system of the vehicle 100. The power train control unit402 monitors sensors (such as fuel injection sensors, wheel speedsensors, exhaust sensors, etc.) and uses control algorithms to control,for example, fuel mixture, ignition timing, variable cam timing,emissions control, a fuel pump, an engine cooling fan and/or a chargingsystem.

In the illustrated example, the vehicle data bus 404 communicativelycouples the sensors 102 a-102 e, the cameras 104 a and 104 b, theinfotainment head unit 106, the on-board computing platform 108, and apower train control unit 402. In some examples, the vehicle data bus 404includes one or more data buses. The vehicle data bus 404 may beimplemented in accordance with a controller area network (CAN) busprotocol as defined by International Standards Organization (ISO)11898-1, a Media Oriented Systems Transport (MOST) bus protocol, a CANflexible data (CAN-FD) bus protocol (ISO 11898-7) and/a K-line busprotocol (ISO 9141 and ISO 14230-1), and/or an Ethernet™ bus protocolIEEE 802.3 (2002 onwards), etc.

FIG. 5 is a flowchart of a method to map obstacles behind the vehicle100 of FIG. 1 while the vehicle 100 is backing up, which may beimplemented by the electronic components 400 of FIG. 4. Initially, atblock 502, the data fuser 116 waits until the gear selection is switchedinto reverse. At block 504, the data fuser 116 obtains images from therear-view camera 104 a. At block 506, the data fuser 116 performsfeature detection on the images to identify and/or classify the objectedin the images. In some examples, the data fuser 116 also estimates thedistance between the vehicle 100 and the detected object(s). At block508, the data fuser 116 obtains measurements from the vehicle locationsensors 102 d-102 e. At block 510, the data fuser 116 determines theposition of the vehicle 100 and the position of the FOV of the rear-viewcamera 104 a relative to the three-dimensional space around the vehicle100. At block 512, the data fuser 116 creates a point cloud of thethree-dimensional area behind the vehicle 100 based on the images fromthe rear-view camera 104 a. At block 514, the data fuser 116 create asuperimposed image with the images from the rear-view camera 104 a andthe point cloud of the three-dimensional area behind the vehicle 100. Insome examples, the superimposed image is describes in connection withFIGS. 2 and 3 above.

At block 516, the data fuser 116 determines whether the point cloud isreliable. The point cloud is reliable when a threshold number of imageshave been analyzed and/or when the rear-view camera 104 a is notobstructed. When the point cloud is reliable, the method continues atblock 518, otherwise, the method returns to block 502. At block 518, thedata fuser 116 displays, via the center console display 114 of theinfotainment head unit 106, the superimposed image. At block 520, thedata fuser 116 determines whether any of the detected objects match anyknow hazards (e.g., the pedestrian 112, animals, bicycles, etc.). If anyof the detected objects match a known hazard, the method continues atblock 526. Otherwise, when none of the detected objects match any knownhazard, the method continues at block 522.

At block 522, the data fuser 116 determines whether changes in the pointcloud between subsequent analysis of images from the rear-view camera104 a indicate that one or more objects are in motion. In some examples,When one or more objects are in motion, the method continues at block526. Otherwise, when none of the objects are in motion, the methodcontinues at block 524. At block 524, the data fuser 116 determineswhether the vehicle 100 is within a threshold distance (e.g., 6 inches,1 foot, etc.) of an object based on the point cloud and the location ofthe vehicle 100 determine using the measurements from the wheel speedsensors 102 d and the wheel angle sensors 102 e. When vehicle 100 iswithin the threshold distance of an object, the method continues atblock 526. Otherwise, when the vehicle 100 is not within the thresholddistance of an object, the method continues at block 528.

At block 526, the data fuser 116 provides an alert to the driver via theinfotainment head unit 106. In some examples, the warning is anaudiovisual warning (e.g., a sounds, a textual warning, etc.). In someexample, the warning is different color coding for portions of therepresentation 212 of the cross-section of the point cloud (e.g., yellowfor portions that indicate motion, red for hazards and/or closedobjects, etc.) or outlines 306 that represent the boundaries of theobjects. In some examples, the data fuser 116 estimates the speed andtrajectory of the moving object and displays an indicator (e.g., amessage, a arrow showing the projected trajectory, etc.) when theestimated speed and trajectory places the object behind the vehicle 100.At block 528, the data fuser 116 does not provide a warning. At block530, the data fuser 116 determines whether the vehicle is moving forwardat a speed greater than a threshold speed (e.g., 5 miles per hour, 10miles per hour, etc.). When the vehicle 100 is traveling at a speedgreater than the speed threshold, the method ends. Otherwise, then thevehicle 100 is not traveling at a speed greater than the speedthreshold, the method returns to block 502.

FIG. 6 is a flowchart of a method to map obstacles behind the vehicle100 of FIG. 1 while the vehicle 100 is backing up, which may beimplemented by the electronic components 400 of FIG. 4. Initially, atblock 602, the data fuser 116 waits until the gear selection is switchedinto reverse. At block 604, the data fuser 116 obtains measurements fromthe vehicle location sensors 102 d-102 e. At block 606, the data fuser116 determines the position of the vehicle 100 and the position of theFOV of the rear-view camera 104 a relative to the three-dimensionalspace around the vehicle 100. At block 608, the data fuser 116 obtainsimages from the rear-view camera 104 a. At block 610, the data fuser 116performs feature detection on the images to identify and/or classify theobjected in the images. In some examples, the data fuser 116 alsoestimates the distance between the vehicle 100 and the detectedobject(s). At block 612, the data fuser 116 obtains images from theside-mirror cameras 104 b and/or measurements from the range detectionsensors 102 a-102 c. At block 614, the data fuser 116 creates a pointcloud of the three-dimensional area behind the vehicle 100 based on theimages from the rear-view camera 104 a. At block 616, the data fuser 116create a superimposed image with the images from the rearview camera 104a and the point cloud of the three-dimensional area behind the vehicle100. In some examples, the superimposed image is describes in connectionwith FIGS. 2 and 3 above.

At block 618, the data fuser 116 determines whether the point cloud isreliable. The point cloud is reliable when a threshold number of imageshave been analyzed and/or when the rear-view camera 104 a, theside-mirror cameras 104 b, and/or the range detection sensors 102 a-102c are not obstructed. When the point cloud is reliable, the methodcontinues at block 518, otherwise, the method returns to block 602. Atblock 620, the data fuser 116 displays, via the center console display114 of the infotainment head unit 106, the superimposed image. At block622, the data fuser 116 determines whether any of the detected objectsmatch any know hazards (e.g., the pedestrian 112, animals, bicycles,etc.). If any of the detected objects match a known hazard, the methodcontinues at block 628. Otherwise, when none of the detected objectsmatch any known hazard, the method continues at block 624.

At block 624, the data fuser 116 determines whether changes in the pointcloud between subsequent analysis of images from the cameras 104 a and104 b and/or measurements from the range detection sensors 102 a-102 cindicate that one or more objects are in motion. When one or moreobjects are in motion, the method continues at block 628. Otherwise,when none of the objects are in motion, the method continues at block626. At block 626, the data fuser 116 determines whether the vehicle 100is within a threshold distance (e.g., 6 inches, 1 foot, etc.) of anobject based on the point cloud and the location of the vehicle 100determine using the measurements from the wheel speed sensors 102 d andthe wheel angle sensors 102 e. When vehicle 100 is within the thresholddistance of an object, the method continues at block 628. Otherwise,when the vehicle 100 is not within the threshold distance of an object,the method continues at block 630.

At block 628, the data fuser 116 provides an alert to the driver via theinfotainment head unit 106. In some examples, the warning is anaudiovisual warning (e.g., a sounds, a textual warning, etc.). In someexample, the warning is different color coding for portions of therepresentation 212 of the cross-section of the point cloud (e.g., yellowfor portions that indicate motion, red for hazards and/or closedobjects, etc.) or outlines 306 that represent the boundaries of theobjects. At block 630, the data fuser 116 does not provide a warning. Atblock 632, the data fuser 116 determines whether the vehicle is movingforward at a speed greater than a threshold speed (e.g., 5 miles perhour, 10 miles per hour, etc.). When the vehicle 100 is traveling at aspeed greater than the speed threshold, the method ends. Otherwise, thenthe vehicle 100 is not traveling at a speed greater than the speedthreshold, the method returns to block 602.

The flowcharts of FIGS. 5 and 6 are representative of machine readableinstructions stored in memory (such as the memory 408 of FIG. 4) thatcomprise one or more programs that, when executed by a processor (suchas the processor 406 of FIG. 4), cause the vehicle 100 to implement theexample data fuser 116 of FIGS. 1 and 4. Further, although the exampleprogram(s) is/are described with reference to the flowcharts illustratedin FIGS. 5 and 6, many other methods of implementing the example datafuser 116 may alternatively be used. For example, the order of executionof the blocks may be changed, and/or some of the blocks described may bechanged, eliminated, or combined.

In this application, the use of the disjunctive is intended to includethe conjunctive. The use of definite or indefinite articles is notintended to indicate cardinality. In particular, a reference to “the”object or “a” and “an” object is intended to denote also one of apossible plurality of such objects. Further, the conjunction “or” may beused to convey features that are simultaneously present instead ofmutually exclusive alternatives. In other words, the conjunction “or”should be understood to include “and/or”. As used here, the terms“module” and “unit” refer to hardware with circuitry to providecommunication, control and/or monitoring capabilities, often inconjunction with sensors. “Modules” and “units” may also includefirmware that executes on the circuitry. The terms “includes,”“including,” and “include” are inclusive and have the same scope as“comprises,” “comprising,” and “comprise” respectively.

The above-described embodiments, and particularly any “preferred”embodiments, are possible examples of implementations and merely setforth for a clear understanding of the principles of the invention. Manyvariations and modifications may be made to the above-describedembodiment(s) without substantially departing from the spirit andprinciples of the techniques described herein. All modifications areintended to be included herein within the scope of this disclosure andprotected by the following claims.

What is claimed is:
 1. A vehicle comprising: a display; a rear-viewcamera; and a processor to: generate a three-dimensional model of spacebehind the vehicle based on images from the rear-view camera; generatean overlay based on the three-dimensional model, the overlay includingrepresentation of objects not in a present field of view of therear-view camera; and display, on the display, the images from therear-view camera and the overlay.
 2. The vehicle of claim 1, including arange detection sensor, and wherein the processor is to generate thethree-dimensional model of the space behind the vehicle based on theimages from the rear-view camera and measurements from the rangedetection sensor.
 3. The vehicle of claim 2, including side-view mirrorcameras, and wherein the processor is to generate the three-dimensionalmodel of the space behind the vehicle based on the images from therear-view camera, the measurements from the range detection sensor, andimages from the side-view mirror cameras.
 4. The vehicle of claim 1,including vehicle position sensors, and wherein the processor is totrack a location of the vehicle and the field of view of the rear-viewcamera in relation to the three-dimensional model of the space behindthe vehicle base on measurements from the vehicle position sensors. 5.The vehicle of claim 1, wherein the overlay includes a cross-section ofthe model indicating the edges of objects within the model.
 6. Thevehicle of claim 1, wherein the processor is to classify objectsdetected by the rear-view camera.
 7. The vehicle of claim 6, wherein theoverlay includes: boxes around the objects extending into a portion ofthe overlay with representations of the objects not in the field of viewof the rear-view camera; and labels associated with the boxesidentifying the corresponding objects.
 8. The vehicle of claim 1,wherein the processor is to provide an alert via an infotainment systemwhen one of the objects is identified as a hazard.
 9. The vehicle ofclaim 1, wherein the processor is to provide an alert via aninfotainment system when the vehicle comes within a threshold distanceto an edge of one of the objects according to the model.
 10. The vehicleof claim 1, wherein the overlay includes a first indicia demarcatingsurfaces that are not in the field of view of the rear-view camera, anda second indicia demarcating surfaces that are in motion.
 11. A methodto assist a reverse moving vehicle, the method comprising: generating,with a processor, a three-dimensional model of space behind the vehiclebased on images from a rear-view camera; generating an overlay based onthe three-dimensional model, the overlay including a cross-section ofthe model representing edges of surfaces within the model; displaying,on a center console display, the images from the rear-view camera andthe overlay; and providing an alert via an infotainment system when (i)one of the surfaces in the model is identified as belonging to a hazardor (ii) when the vehicle comes within a threshold distance to one of thesurfaces according to the model.
 12. The method of claim 11, whereingenerating the three-dimensional model of the space is also based onmeasurements from range detection sensors; and wherein thethree-dimensional model of the space also includes areas to the side ofthe vehicle observable from the range detection sensors.
 13. The methodof claim 12, wherein generating the three-dimensional model of the spaceis also based on the images from side-view mirror cameras, and whereinthe three-dimensional model of the space also includes the areas to theside of the vehicle observable from the side-view mirror cameras. 14.The method of claim 11, including tracking a location of the vehicle anda field of view of the rear-view camera in relation to thethree-dimensional model of the space behind the vehicle base onmeasurements from vehicle position sensors.
 15. The method of claim 11,wherein the overlay includes a first indicia demarcating the surfacesthat are not in a field of view of the rear-view camera, and a secondindicia demarcating the surfaces that are in motion.